fix the UGI problem when reading ORC files#504
Closed
VicoWu wants to merge 1 commit intoapache:spark2from
Closed
Conversation
Contributor
|
What's the JIRA number for this. So that GIT PRE-COMMIT testing will be done automatically. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As mentioned in SPARK-11248, the spark thrift server have security bugs , cause the result that user A sometimes have the authority of user B and User B sometimes have the authority of user A in turn. I debugged it and I find that it is caused by the hive 1.2.1 library , OrcInputFormat.java, in which a thread pool is created to contact with remote HDFS. Since threads in pool is reused and shared, so , when thread-1-pool-1 is used by user A previously and after that user B is assigned to this thread in coincidence, then user B will have the security context of User A.
I have fixed this bug by add UserGroupInformation in this pool, to make sure that when a user is assigned a thread, then the security is switched to this user at the same time.